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INTRODUCTION 


The WO Model of Kruger and Norrisl was developed to provide an approach to the optimization 
of a test program b^d on prediction of flight performance with a single flight option in mind, and 
nly mm^al consideration of reflight was made. In this paper we extend the process to more than 
one flight as m Space Shuttle missions. We utilize the concept of “utility” which was first 
developed under the name of “availabiUty” by Bloomquist,2,3 and further developed by Kruger 

a model developed by Williams and Kruger^ is 

used to follow i^rformance through the various options that one encounters when one has the 
reflight and retnevabihty capabilities of Space Shuttle. 

Value” model proposed by Kruger and Norris is modified to produce a measure of the 
probabihty of a mission s success, achieving a desired utility using a minimal cost test strategy. The 

resultmg matrw of probabilities and their associated costs provides a means for project man^emen» 
to evaluate various test and reflight strategies. nidnagemcn. 

Finally, recommendations for future study are provided. 


THE CONCEPT OF UTILITY 

tried and standard concept 

Sm f ‘. 4 ^* tn part, in the mathematical theoretical development of the 

on S^crShSttte^ developing complexity of space payloads (sateUites, experiments 

As mentioned in Gnedenko, et. al. 5, “one of the most intriguing problems in rehability theory is 
of o/ design of a complex apparatus that will function even when some 

”®t. Biological systems possess this valuable property to a high degree The 
study of biological systems from the point of view of the principles of their desi^ and^h reliabU- 

technological accompUshment. We are convinced 
that nature has t^en a course not only along the lines of extravagant standby redundancy but 
pnmMly tltfough selection of optimum system solutions, i.e., a careful choice of elements capable 
of maintaining an extraordmary stabiUty in performance. No doubt, study of a peculiar feature of 
biolopcal systems from the standpoint of reliability theory wiU enable researchera to discover 

^maples not yej conceived since we tend to approach technical problems exclusively from the 
pomt of view of “traditional technology. ” r v u «« 

As spacecraft become more complex, their nature approaches that of a biological system It is with 
this view m mind that utility models the behavior of complex spacecraft pay toarb^ttcTthan 
rcuabihty. Even though utiUty can be classified as a reliability concept, it is not the same as the 
o^ltiiroTr u* ClassicaUy, reliabiUty is thought of as the probabiUty of failure-frcc 

UtiUty is a mcasuTe of the Overall 

^ftilnew of a spacecraft or a payload and U thought of as the successfulness of a mission as 
® perfect ^ssion. Thus it allows for multiple failures to occur during the flight. 

Ut^ty does not ded with the probability of the faUure free operation of a system. As a sequence 

sunilar to how a biological system performs. As failures or aging occurs, biological systems still ^ 
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function, but to a lesser degree than before. With this in mind, it seems like the transitional step to 
the study of biological system is to study spacecraft operation. Although the previous statement is 
somewhat contrary to Gnedenko’s quote, we believe the line of discovery is from simple systems 
to complex systems. 

In spacecraft operation or payload performance, each anomaly can be classified as to its seriousness 
or the amount of degradation that it causes to the particular mission. Utility can then be calculated 
from the observed occurrence of a random sequence of anomalies or types of failures by assigning 
a certain criticality to each of the failures in the sequence and then considering utility of the pay- 
load or spacecraft after the occurrence of any failure as the product of one minus the critic^ity 
tenn at the particular failure multiplied by the previous remaining utility. 

The instantaneous utility U, is defined as follows: 

where Dj denotes the criticality of a type i failure, S2 is an index set for the various criticalities of 
failures that occur during space flight, and nj is the total number of failui es for any particular 
type of criticality during any space flight. 

The average utility, U, is defined in Kruger and Norris as 



where t is the duration of the mission. It should be noted that this definition is a slight modifica- 
tion of the definition of utility given by Williams^ ; this modification arises in the discussion of the 
calculation of criticality. For a complete discussion of the concept of the criticality of failures and 
their classification, see Bloomquist2.3, Timmins^, and Williams^. 


To extend equation (1) to multiple flights, one must consider various options such as repair and 
refurbishment; repair, refurbishment, and retest; and repair or refurbishment. These various options 
and^ their effect on the calculation of utility will generally fall into one of three decision-making 
options. If one repairs perfectly, then the initial (or instantaneous) utility in that particular flight 
reverts back to the maximum value of 1 .0. If one does not repair or retest, then the initial instan- 
taneoM utility of the subsequent mission is related to the instantaneous utility at the end of the 
last mission. If there is testing in the intermediate stage, then the average utility of the proceeding 
mission is influenced by this amount of testing, and the average utility of the mission is changed 
accordin^y. The mathematics of these options and their effect on utility during multiple flights 
will be discussed in the section on Failure Flow Process. 


Calculation Of Utility For Multiple Flights 


To calculate utility for multiple flights, we have a finite sequence of utilities, {UjlJli, which 
corresponds to some particular managerial decision-making process. For example, Dj would be 




^ thT/oT subjecting tne spacecraft to the thermal-vacuum test environment 

onl^tMt* The uthity thus obtained is a result of some testing; the more 

one tests, the more utihty one obtains up to a fixed point less than 1 . 0 . 

«pfta Md SXt d«ir.d >«n,e utiUly, O 4 . one either renys a spacecraft without repair, or 

reflv -mHnn it ^ a managerial option. If, for instance, one selects the 

re^ option, then a second average utiUty, Uj. is obtained. As this decision process continues one 
obtws a sequence of utilities. The argument for the calculation of 0 as affected by multiple flight 

even though it is speculated that most experiments will only experience 

one or two flights beyond the original flight. *u omy experience 

Smee utility can be thought of as the amount of information collected as compared to tlie amount 

mean that it is a measurement of the overall utUity of all of the components before utility is 
» m over .ta. where the number of failures over .ha, .ime in.^aU repl^S 

the average utiUty, then we divide the integral of 

the instantaneous utihty by the length of time interval. We thus have 


D*)*"^*^ dt 


Ti 1 

U = - J at ( 3 ) 

h?h"clSiIIrve“Lct^^^ of of a failure, and F(t) 
continuous failure modes over several flight and reflight options then we mav con- 

f J tt eTfli r schf e viet " f thr 

If rh-f * ^ V *3 be the total time in flight three 

If there were no repair before flight two, then the average utiUty for flight two, is ^ 

Tl ‘ F(t) 

^2*r J (1-D*r‘^dt 

‘l 

If there were repair before flight two, then 

®2*r 
h 0 

** the cumulative failure function when repairs are made. If there were no repairs 
after flights one and two, then the average utUity over flight three would be 


(4) 


(5) 


U, 


(tjtt2+t3) 


/ 

(ti+tj) 


(1 - dt 


( 6 ) 
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and if there was repair after flight two, 


/ ( 1 c 

^3 0 

TTie exact failure inodes F(t) and F^(t) wUl be calculated as m the FaUure Flow Process section. 
We can see that the effect on the limits of integration of a repair is to start the time clock at the 
begmning or to integrate from zero to a final time. 


Certainty of Obtaining a Specified 0 


One of the basic assumptions made in the TVTO model of Kruger and Norris is that a certain aver- 
age utility (termed av^ability in that report) results from a series of tests. If a project manager 
specifies a certain desired level of utility, then a decision-making process that involves cost and 
uncertamty would be necessary. 


Uncertainty means some inability to predict accurately. In the case of testing, this means that we 
are not 100 percent certain that we wUl obtain the utility that we specified. Therefore it is neces- 
sary to calculate the risk involved in obtaining a specified average utility. 

Once the manager has specified the degree of certainty desired, the objective of a cost strategy is 
to <)vUin the maximum performance for the minimum cost. Following this line of reasoning we 
esta. lish minimum cost curves for a given utility. These minimum cost curves are unique to the 
Particular project and have to be derived under specific conditions. 


We now turn to the derivation of the probability of obtaining a given U. As mentioned in 
illiams , U IS a function of D* (some average measure of criticality) and m (some number of 
failures over a specified time). For any particular flight, the time period is specified. For example 
various nussions were investigated in Kruger and Norris. 


It is important to note that ni is represented by a function of the form 


Fo(t)«Aot®o 


in** ^ replaced by some constant value as given in Williams^. This value is, for a speci- 
tiedyu percent certainty, the upper confidence limit for D* which we denote by D,*. . Once D* 

IS replaced by D* at a particular confidence level, C then becomes a function of the fadurcs in 
any particular flight. 

To specify the probabdity calculation of 0 or the risk of 0^0^, where Da is given, we must 
? miVH* question, “Does a given U^j give or specify a unique fadure mode function 
rott) . Lookmg at printouts from the various options in the calculation of utility versus total 
ni^ber of fadures, the answer appears to be affirmative. To formalize the above idea, we state the 
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Theorem (1.0). A given 0^ specifics or gives a unique failure mode function Fo(t). 
Proof : Recall that 







( 8 ) 


B8iv«n with the assumption that 1 th 

p ocess dependent. This assumption will be elaborated upon in the section on Failure Flow Process. 


Assume that 0 is given. This implies 



0 




where C * 0 • tf, A « tf, and K * 1 - D* . 

Letting x = Aq and using these substitutions, we have 



(9) 


( 10 ) 


where 

f(t) = Kt^° . 

Using difference quotients and applying the limit, we have 


F(X,-U2J. 


F'(xi - /* P(.) jtait dt . f tx(„ i„ 

® *" -1 0 


dt 


( 11 ) 


S n« oil 0 is decreasing continuously over time, 

^nce U or F(x) « decreasing over time, this implies that if xj =?fex^,then F(X|)#F(x^) This 
g the case, there are no two different Aq’s which give rise to the'same F(x) Thisin^tum 

impUcsthat Fo(t)*Aot ° is specified for a given 0. 


The imponance of Theorem 1.0 is that it enables one to make probabilistic or risk statements about 

U from the probabUistic structure of the Product Limit estimation procedure for F(t) found in 
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Williams and Kruger*. Before we develop a method for calculating the probability of obtaining a 
specified 0 , we give a brief review of the product limit estimation procedure found in Williams 
and Kruger. 

The Product Limit estimate of the distribution of failures during the component and system level 
tests and the orbital flight of a payload measures the performance of a spacecraft in terms of its 
ability to survive in that particular mode of operation. As in reference (4) we make the following 
designitions: 

(1) l^(t) » n|, the number of failures 

/s 

(2) F*(t) ^nj/N s F(t)/N where N is the number of components in the payload 

(3) P(t)si-Ht). 

The next question we explore is how to ascertain the probability of obtaining a certain utility. 
Suppose from Theorem 1 .0, we have a unique failure mode function which may be obtained from 

the failure flow process. Thus, U = C, which corresponds to F(t) * n or to F*(t) = n/N or to 
^t)»(N-n)/N. 

If we now calculate probability based on the above statements, we have not expressed the variabil- 
ity of the system and are operating at a 50 percent risk level. Another way to visualize this is that 

we are operating on the curve that describes cumulative failures, F*(t), without any variability, or 
we are estimating with point estimates. To deal with this situation, we have one of two alterna- 
tives: 


1) we test and accept less utility at a given level of cenainty 


or 


2) we test more to achieve a higher level of utility at the given level of certainty. 
Note, a test-fix process is assumed. 


In order to develop a scheme to accomplish either of these two above objectives and to observe the 
variability of 0 , we refer to Williams and Kruger and derive the appropriate probability state- 
ments. An example will help visualize the approaches. 


EXAMPLE I 

Suppose tj-0.40 corresponds to or is equivalent to F(t)=15. This implies that F*(t)* 15/16, 
where N * 60 ; this in turn implies that P(t) * 45/60 . 

Using equation (12), reference (4), we find that 

F*(t)-l+(Fnt)-l)[l±Z^^2 (1 
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or equivalently: 


F*(t) - F*(t) ± (F*(t) - 1 ) (Z^/2 > V V(t) 


(13) 


Suppose that y^) ■ 1/32 for sake of this example. Inserting the appropriate values into the 
above equation at the 95 percent confidence level, we have 


F(t)= 60 (0.205, 0.295] =[12.3,17.7] . 


This means that we are 95 percent confident that we do not have any more than 17.7 failures. This 
also translates into a statement about utility. If we want a 95 percent bound on utility, we have to 
relate 17.7 failures to utility. This may be done by solving for A© in the general failure process 
model in the orbital case or by searching for the appropriate number of corresponding failures for 
a given utility on the failure flow process program. After this is done, we see that the corresponding 
utility is less than the utility we achieved. 


EXAMPLE 11 

The other approach is to specify a desired utility, and then test»fix beyond that level to obtain the 
desired utility at the degree confidence that the project manager specifies. This may be illus- 
^ted as follows: Suppose Uu = 0.40 and one tests beyond this level to obtain some utility say 
U2*0.50. 

As in the previous example, we obtain a ^nfldence interval for the number of failures that 
correspond to the particular bounds on U , say (6, 12), with the intcival centered at 9. Now 
suppose that 0 * 0.40 coiresponds to 1 1 failures. To find Pr(U <0.40 | Oj = 0.50), the probabil- 
ity that U falls below 0.40 given that one has achieved a utility of 0.50, one has to calculate 

/V 

Pr(F(t) > 1 1 I F(t) * 9). This is a measure of the risk of not obtaining 0 = 0.40 , given that one 
has obtained 0 * 0.50 through a prior testing program. Thus, the probability of obtaining at least 
0 « 0.40 is Pr(F(t) < 1 1 | F(t) = 9). 

To formalize the exatnples in terms of U's, we consider equation (13). Let this confidence interval 
correspond to, say^U, "^1 • To make a probability statement about O^j «C 2 given that D, 

•C^ where < U, , we have to relate 0, " Cj to a given number of failures and then to 
Fj(t) = nj /N by dividing the number of failures by the number of components. Once this has 
been done, we must relate « C 2 to Fljft) ■ nj/N . 


7 


After these relationships are made, then 


Pr(U^>C2 I ■C,)-Pr(Fj(t)<n2/N| F*(t)-nj/N) 


«Pr 


■ ^F;(t)-F5(t) 

^a/2 ^ T: 7 ^ 

[F;(t)-n 


( 14 ) 


We have a pseudo-normality situation; that is, a situation where one can make probability state- 
ments about a ^ven vancble through the normaUty of another related variable. For purposes of 
discussion and diagramatic arguments, we visualize this symbolically as 


U, - n (U^j. o(U,)) 
^p.s.n. * "■ 


(15) 

(16) 


where Zp , „ stands for the pseud^normal random variable and is calculated by the formal given 

^ calcutoted from the Product Umit estimation procedure as described 
m Williams and Krugcx^. As seen by equation (14), it is necessary to calculate or estimate the 

this, we make use of a random number generator and use a Monte Carlo 
technique which we describe in the following section. 


Monte Carlo Estimation of V(t) 


Using methods found in Kruger and Norris, we may determine the unique number of failures for 


a given utility. This gives one Fg(t)aAt ° 
in time, we consider the following example. 


To illustrate how we use this model to place failures 


S.?f **“* ^o^Ponds to the situation where one has five failures over a specified 

Jj* ;• ^ time interval for the orbital 

case. Essentially, we equate Mw«iwuai 



Thus, we have produced a sequence of times which we can use in the Product Limit estimation 
process to calculate V(t) (see equation (10) in Williams and Kruger). 
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The above method iUustrates what we do in the Monte Carlo estimaiion procedure. In the case 
where we have five failures, we use a uniform random number generator and obtain five numbers 
on the interval (0, 1 ). Suppose that we obtained, after ordering, the sequence, (0.2. 0.34, 0.52, 

0.6, 0.82). We next multiply these numbers by five and obtain ( 1 , 1.7, 2.6. 3, 4. 1 ). Letting 

ni » I . n2 ■ 1 .7, n3 « 2.6, 04 - 3, and 05 - 4.1, we calculate tj (5 with the above formula for 

» •••* A, 

After obtaining the failure times, we use the Product Limit estimation procedure to define the 
parameters necessary to calculate Vjit). We repeat the process and calculate Vjlt). After nsccps, 

n 

V(t)-yV|(t( (,7) 


By the central limit theorem and Monte Carlo techniques, V(t) converges to V(t) . Since the 
generalization of this example is apparent, we do not describe it here. 

To conclude this section, we may note that the mathematical procedures for the option "we tes, 

and accept less utility at a given level of certainty” is very similar to the derivation described above 
and wUl be omitted. 


APPROACH TO CHOOSING A COST 

To adapt the preceding discussion of utility to a cost model, it is beneficial to evan.ine tne environ- 
manager. Increasingly, budget consideration will constrain testing and 
refurbish/reflight decisions. Furthermore, there very likely will be competition among a variety of 
projects for these limited resources (see Sayles and ChandlerS). The allocation of resources - 
the^nager*^*"’"* or personnel - among the available alternatives wiU be a growing problem for 

The fiscal constraints play a large role in project management (see Lloyd and LipowP). To date 
most spacecr^t test cost optimization models have relied on expected value concepts, e.g. 
Campbellio, Donelwnl 1 , and Naegele and Sellinscheggl 2 . while incorporating the range of 
possible costs, the single expected value - which in some models is supplemented with a measure 
ot vanabmty - may not be of great help to a manager facing budget limitations. At times the 
opt^al test strategy may not fit into the available budget, or other needs are competing for funds 
makmg a non-optimal solution necessary. The suggested format of the modified cost model clearly 
dBptays the marginal changes in performance which may be purchased ''or additional test dollars, 
ihe tormat provides probabilities of success for various levels of investment in testing. 

Given the few repetitions of an experiment available to most project managers and their need to 
know close-to-actual cost, we have modified the expected value model in this report. While con* 
***IJ?? ? probabiUstic mathematical approach of expected value, the output of the 

modified cost model provides probabilities of success for various levels of costs. The model com- 
putes the least cost test and refurbish/reflight strategy that will attain a particular probability of 

achieviag desired utility. An example of an output matrix for a specified, desired utiUty is shown 
in Taole I. 
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Table I 


Probability of Achieving 0 = 0,53 Versus 

Least Cost Strategy 

Probability of achieving 

Cost of least cost 

U » 0.53 

strategy 

60% 

50,000 

70% 

65,000 

75% 

73,000 

80% 

85,000 

85% 

97,000 

90% 

112,000 

95% 

153,000 


w«Sd'« manager or research team has determined that a utiUty of 0.53 

1 provide sufficient data for mission success (for a discussion of determining U,< from oast 
spacecrait performance, see WiUiams and Kruger^). The least cost strategies fw obtaining^the 
probabilities m the left column for the particular spacecraft’s characteristics - comptexity of 
desi^ weight, volume, refurbishment costs, flight parameters, and extra STS services - are cal- 
culated and !.at,d in the right column. With thia information Jhc proicct m^ag«rabl. to 

1 . how much a certain probability of success will cost, 

2. the highest degree of certainty affordable within the project budget, and 

taw additional investments of specified dollar amounts wiU enchance the probability 

of s^^. or conversely, how specific dollar amounts debited will reduce the 
probabihty of success. 

tatwM^lf Table 1 is that a manager may make cost comparisons 

„ fh f ♦ program. If a program has a number of experiments competing for 

resources, the format presents information that can be used for allocating fLds among thL For 

a Z“;?orer"rbTbT.f“?' “ T 

LoHed ^ ^ ^ ^ Of success; the funds freed by this reduction in spending may then be 

apphed to another experiment enhancing its likelihood of success. 


3. 


Certainty of Achieving a Desired Average UtiUty 

The design of the modified cost roodd relies on the assumption of a near normal distribution of 
utihty and on a distinction between Og. desired average uriUty, and u!, acW^lle “^2 uMIltv 

’ kx**'* *• tacome common to speak of C, as if there was a S(K 

^habrirty of actually attaining that level of utility or one higher. Through 7he 0 tami 

mill, be to a higher utility, again with a .50% probability of that utility or a higher one 

being mached. Given ,h. roughly normal character of the distributions of “^. 1 , me^^s Z, 
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Figure 1 


The Effect of Testing on Utility 
After Testing, Uj>Uj 


In figure 2a, there is a 5(^ probability of attaining U«'. 
probability of attaining Uj" or higher, where Uj< Uj . 


After testing and repairs, there is a 50% 


ity one^aS **«' 50% probaba. 



Figure 2 

Post-Test Probability of Attaining at Least Uj. 

israTr r 

aWfts w«?mMns™a*< 0'^°0"^ Program progresses, the utility distribution - 

™ wun means Uj < U, < Uj < ... . Each step signifies improved utility or oavload r 
fonnaoc. tluough more tho«>ugh testtag and correction of identSed ' 

' J* * P»y*Odd to the point where Pr(Uj) - S or U, • On With 

further testmg and corrections, there wiU be increases in U As ff\nd fr ^ ‘ u ^ 

accompanying increases in Pr(0, > 0„ nuty be «“™^d Ss fotoi^?: '' 


1 curve 
1 per- 


II 


0 ?: ’ “-'O ^ < 0) whe« PKZl ^ 2 < 0, is .h. probability 


2 , = 


^d-Ua 


• SpacmlSuT " «e action on Carulmy of Obbtwns 

1. the existence of a U„,ax term for various flights and 

concepts, we consider figure 3. ^ improve Ug . To help visualize these 



Figure 3 

The Effect upon Test Program Resources of Incremental Changes in Og 
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From figure 3 we see that additional investments, Sj , $ 2 , and S 3 , in testing to increase 0, produce 
decreasing gams m 0», A, . A 2 , and A 3 . Eventually these ga_ins approach an asymptote where 
T? j*"; n* “PP" improvement in Ui that is possible through testing. 

lannlinahU « 

mapphcable, and therefore, even if this test were perfect, a certain number of failures remain and 

give rise to an average utiUty of less than 1 . 0 . 

Increases in 0, are purchased through testing and corrections. Therefore, we may assume a rela- 

S ® uniformly increasing function with time, where cost 

figure T^*"*** average utility, 0 ^, is the abcissa, we may visualize the minimal cost curve as in 


s 



Figure 4 

Hypothetical Minimum Cost Curve 

,^* .*^* ^ minimum cost curve for attaining the various average 

utilities. S « the mmimum cost test strategy to achieve 0^ = 0^. The minimum cost curve is 

^nln? y TVTO computer model output. The output provides mixes of cL- 

ponent and system level tests and the cost and improvement in Ua for each mix. The model takes 
mto account payload complexity when making cost estimates. 


Extending the Model to Multiple Flights 


To this ^int, the cost model has mcluded only ground-based testing - mixes of component and 

TvS 7- SP®®® Shuttle, reflet becomes 

a viable alternative. There are three situations in which the refurbish/refiy strategy must be 


1. if the desired Uj is greater than U^ax' 

* IhnU* ^d via ground baaed testing is beyond the project’s budget 
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3. if the cost of attaining O^j with ground base testing alone is greater than the cost of 
refurbishing and reflying the payload. 

Examining these situations separately will help to disclose some of the features of the refurbish/ 
refly option. 

In the first situation, when Uj > » unlimited resources devoted to testing will not yield 

the necessary probability of achieving . In large part this is due to modes of failure unrelated 
to the test program. 

After the first flight and succeeding flights, these modes of failure may be detected and corrected. 
Identifying and correcting flight failures will increase the payload’s performance capability upon 
reflight. 

It is unlikely that payloads will be flown solely for engineering performance information. Rather, 
they will perform their data collection or transmission duties as well as have their engineering 
performance monitored. The collection of data on the first flight may reduce the data collection 
needs of the second flight. The second flight may h^e less stringent performance requirements 
and yet result in an overall successful mission. The per flight therefore, may be reduced 

througli a flight/refurbish/r^y strategy. A reduced on the first flight will make it easier 

(possible) to attain a given on the second flight. 

The second situation is the case of budget limitation preventing the desired U^j from being 
reached. If the refurbish/refly option lies within the budget limits, it should be examined. 

This case could exist when the payload has low launch and refurbish cost (low volume and 
weight and easily reproducible parts) but high test costs (many components or systems 
requiring individual testing). As in the first situation, the combination of a more thorough 
“test” sittwtion and tlte partial collection of data may provide an increase to the necessary 


In the fu^t case it was suggested t^t tlie refurbish/refly strategy, by providing the “ultimate 
test” situation, may increase the available by reducing the number of non^thennal- 

vacuum related failures that a payloao will encounter. Estimates from a model have the 
potential to show how much ground*based testing simulates the actual conditions of space 
flight (in terms of the number of failures detected). The refurbish/refly strategy, by using 
the space environment, reduces the area of uncertainty; the potential for successful performance 
is thereby increased. Figure 5 illustrates this increase in U,jjax of a 

second flight option. 
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Figure 5 

The Effect of Refurbishment and Reflight 

Cu^e A in figure 5 depicts the improvements in average utility derived solely from ground-based 
testmg. It IS a testing, not a testing and reflight, strategy. ground-based 

C'^B represents the cost curve for a test strategy that combines ground-based testina with 

ground-based testmg. Curve B represents the costs to refurbish and refly a spacecraft once olus 
^ou^evels of pound-based testing. As ground-based testing is inc7e»e^, l^co^ 

sS e^A "“.r ** oPPortunities'to detecUaSto 

optiom ’ “mas . the maxunum average utility achievable for a particular 

The retation of curve A to curve B wiU be determined largely by payload size and comolexitv 
taunch costs are a function of payload volume and weight, the orbit inclination and the flielit 

!hr^avk!S““I‘^' ‘P™ ttPPP" to Oopptitl upon the complexity Md sensitive of 

the paytoad. Another cost v. hich may be significant is the cost of delaying the ^e« hlourt a 
^ond fhght cycle. If a group must be held in reserve while the refUrbKfitStls aoSt i 

^“.eS'rel^ - *ti.hiy^t^tlTa« S 

There may be some concern that the cost curve for strategy B, the single reflight strateev includ#>« 

« ealt onril^rtU^er** although the payload is actuaUy flown twice. Since all missions include 
na rt nf th 1^.**®"** ” Considered common to all missions and is not a 

Li? r of the test costs untfl tL"esr^^^^^^^^^ 

refUehTc^l n"* included as test costs. The inclusion of the 

refhght costs allows comparison of the various reflight and ground-based testmg combinations to 
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to thi. innation factor. Fto **“ <“« 

B, jmt t«tin, or tcstta* Ld^nytaT ” “““ 



^ '-umpanson oi strategies 


i~k UilVi iJ 


( 18 ) 


If the flights include identical services anrf fh i ® occurs before the flight in A. 

flight 1 or B wm be weight and volume is not different, then 

the flight 2 in B. An earUer flight wUl be less rastiv f expensive than 

savings accruing to earUer flights may be estiLated^af mflation) than a later one. The 

Savings = L(1 + E)^A . ^ ^ 

Where L » present flight costs 
E « monthly cost escalator 
= number of months until flight in A 
Jfi * number of months until flight in B 

UsU <te?J“attonX T”^‘ ‘ B. 

Of each strategy calculated computed and the total savings (or loss) 

not fuUy cover of f ‘’«ent that the price chanptd for a tetvice does 

i^c«»» a„ difncult to fore«e. the cost,rncorpo7ted^^K:L■ 
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Estimating refurbishment costs is difficult. The research done to date provides only rough estimates 

Tf th^o^l?. refurbishment of LAnUt vary iLm 30% 

^e "‘'hf * ® variations in costs appear to depend on the complexity of 

LANdS^T reLbUhm time and whether parts are standardized. A re-examination of the 

retu^S^hil;"^ relationship between complexity and 

Retting to fi^re 5, curves A and B are assumed to be minimum test cost curves which incorpor- 
ate the proper flight costs, escalators, and refurbishment costs. Figure 5 may be modified further 

Figure 7 illustrates the three strategies; A, B, and C. 



max A '^max B 
Average tJtility 


max C 


Figure 7 

A Comparison of Three Strategies 


Notice that Omax c « greater than either U^ax a or U^ax b- 

SS'.t C '?■ "“r' ««‘n< »«<1 «wo refurbish/reny 

cycles, strategy C, it should be mentioned, assumes three flights total. 

StraWjiM B and C may provide lower COSB methods for obtaining a given Ud. For some Dd 

Stir.r.h?d “™d“n“ ® »''«‘l“l’le. This is particularly 

As the costs increase rapidly for addiiLalwr 

S ul strata^ c ’ approaches the masiraum available throu|h strata, y B. it may be less costly 
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The model may be extended by adding strategies using increasing numbers of flights. Such exten- 
sion, however, yields smaller and smaller increases in the maximum average utility; that is, the gains 
per dollar of shifting to “higher” strategies decrease. Figure 8 portrays the model extension and the 
convergence of the U„,ax ntany strategies. 



Figure 8 

The Convergence of Attainable 
Average Utility Towards a Maximum 

The decreasing nature of the gain in U^ax values results in a limit which is the ultimate U„,ax 
attainable after many flights. This value, max(Ujnax)» ** assumed to be less than 1.0. That the most 
certain probability of attaining success is less than 1 .0 reflects the possibility of a random failure 
occurring at any time. Max encountered in Space Transportation System by the 

Orbiter itself, its payload bay mecMnisms, etc. after several years of operation. This value might 
also be approached by payloads that are flown repeatedly. 

Given a payload, its volume, weight, flight needs, complexity, and the utility desired to have the 
mission perform successfully, a unique set of minimum test cost curves can be produced. These 
curves will relate gains in the average utility from ground-based testing as well as through refurbish- 
ment/reflight strategies. From the lowest cost frontier of the curve set, the scallop shaped line of 
figure 9, cost and levels of average utility may be matched. The output format with which this 
section began may thereby be derived. 

The merits of using a ‘Lost Value” model as opposed to th>' “Utility vs. Cost” approach should be 
discussed for two different kinds of payloads. 
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Figure 9 

Variation of Cost with Strategy 

If one launches a communications satellite in which the transmission of data can be measured as 
having value^m terms of cost of data obtained and there is a one-time flight situation, then the 
“Lost Value” model or approach developed in Kruger-Norrisl is adaptable as well as the approach 
developed in this section. The difference in the two approaches is that the Kruger and Norris 
Lost Value function takes the decision out of the projects manager’s hands. It chooses the optimal 
test procedure for the manager. 


If one launches scientific payloads where data has a tremendous value and there is a one-time flight, 
then the above comments are applicable to this type of payload. 

On the other hand, if one has reflight capabilities for communication satellites or scientific payloads 
where data is highly valued, then the approach where one considers “cost vs. utility” applies. This 
approach also applies to the situation where one has scientific payloads where the value of the 
data obtained is not tremendous or unknown regardless of whether one reflys the payload or not. 

To conclude this section, we remark that the mathematical procedures for the option, “we test 
to where we are and accept less utility at a given level of certainty” is very similar to the derivation 
described above and will be omitted. 
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ITHE FAILURE FLOW PROCESS 

In order to develop an approach that would extend to multiple flights, we use the logic as suggested 
m Kruger and No^ms . A few sunpUfications were made in the general model, and before discussing 
the new approach, we discuss the basis for simpUfying the model. 


Rationale for Simplifying the Kruger and Norris TVTO Model 

We now turn to the derivation of failure flow analysis and the modifications to utUity under various 
decision entenon. The model developed by Kruger and Norris describes failure occurrences during 
the thermal-vacuum test procedure (accounting for effects of temperature cycling and temperature 
on the overall distribution of failures and normalized for the number of components) as 


F 

N 




+A(l-y)® 



Z] 

pO -y) 

i 


KT + 7)® 


(19) 


To determine the various constants in equation (19), data from 109 component level tests were 
norm^ized, and iteration and fitting procedures vere used to determine the parameters. The pro- 
gram that normalized the data and give the fitting procedures are given in reference ( 1 ). 

A rnethod of estimation of reUabiUty growth was developed by Wmiarns and Kruger^ using the 
Product Limit estimation procedure of Kaplan and Meierl^. This method has the statistical 
property of consistency. This property is a convergence property; by using the Product Limit 
estimation procediue, one has a theoretical, statistical representation of the distribution that 
describes the frequency of failures over time allowing for the adjustments, such as components 
entering and leavmg the flow of tests, that were made to the data in the Kruger and Norris study, 
hen one chooses six-hour increments and estimates the growth parameter B, using the Product 

Limit estimation procedure, one obtains B = 0.478 , and X » 0.0 1 7 for the model F(t) = Xt^. 

When one uses the normalized data from Kruger and Norris and does a fit to the above model, 

one obtams Bj 0.485, and X » 0.018. The multiple correlation coefficient for this data is 
R * 0.978, which means that the model accounts for 97.8% of the variation in the system. From 
tlm result, one concludes that the normalization process in Kruger and Norris and a simple curve 
fit yield the same results for the estimation of B as the Product Limit estimates. In this sense 
one may conclude ttot th^e normalization process for F/N found in Kruger and Norris yields the 
same estimates for B and X as the process for Product Limit estimation or vice versa. 


The Test Process 


An approach to conceptualizing how failures are uncovered may be seen in Heuser* 5 where failures 
are visualized as being detected or escaping tests. In this same way, it is helpful to look at and 
ex^me vanous assumptions and procedures that influence the flow of failures in thermal-vacuum 
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The objeciive of any test program, whether it be environmental or reliability test screening, is to 
discover and eliminate failures that would occur in operational use. To accomplish this goal one 
tnes to conceive all possible environments to which the product would be subjected and then to 
create tests that sunulate these operational environments. Decisions are also made as to some 
optimized test procedures in terms of cost, the quaUty of performance, the risk involved, the 
priority of the spacecraft, etc. 


These kinds of decisions are made for thermal-vacuum tests recognizing that there have been pit- 
inous ***** level and several other levels (see Heuser* 5). Assuming the testing procedure 

has passed through a sequential stage like Heuser’s description of Failure Flow Analysis, one 
encounters questions such as how much testing and what combination of testing needs to be done 
at the thermal-vacuum test stage. 

To help guide these decisions, one needs to know the relationships between the various test stages. 

procedure can be viewed as sequential in nature; that 
«, Inures now (i.e., if not uncovered proceed) from the component level tests to the systems 
level tests and on to the orbital flight. Fmther, the Kruger and Norris model assumes that by 
mcreasmg test duration, one increases the likeliliood of uncovering an incipient failure. With these 
assumptions, one would like to know how long to test at the component level to eliminate a 
reasonable amount of those failures which can be uncovered at that particular stage of testing. 

This decision must be bwed on cost and time considerations and on the future quality or perform- 
Mce of the spacecraft. To estabUsh the cost-quaUty relationships in the thermal-vacuum area, it is 
necessary to model the faUure mode, the cost mode, and the quaUty as testing takes place in this 
environment. It should be noted that quality encompasses the concept of utiUty. 

We begin by discussing failure modes. To envision the process we consider the diagram of figure 10. 



Figure 10 

The Flow c .’ Equipment 


From previous testing and quality control of parts, we have screened out a certain number of 
tailures (as appropriate references, we mention Heuser* ^ and A. Krauszl ®). 


I one has a procedure where failures arc repaired when they occur, then it is conceivable (dis- 
regarding wearout modes and random faUures) that by testing over a large enough time interval, 
one could ehmmate those component failures that occur and are detectable during thermal-vacuum 
comjwnent level testmg. One, however, has to consider the amount of testing in the Ught of cost 
wd time constramts and m terms of the reUabUity or, in this paper, utiUty requirements. Since 
cannot achieve perfect utility, then one must specify some level of utility that is necessary to 
meet the desired performance goal of the finished product. 


It is in this framework of cost-benefit analysis that models play an essential role. 
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The Role of Models 


ponentsJnd how identiOes its main com- 
mode!: ^ mieirelated. The followmg are some of the key elements of a 

for easT and cl^ty^^Sden^^ simpUfication is required 

4. The model depicts melitj, for « p,„ic„tar purpose and a particular audience. 

fher m isTo t'tTd r ■« 

to which it leads. ^ Itdity of the conclusions or decisions 

of the process to tS nex^fasTuustrat^^^^^^ ^ P^o^resses from one stage 

a particular cost after a 'cert“i; ™olfof"tS Ll“e. « 


The TVTO Model Equations 

Table II is provided as a guide to the equations that follorv in this section. 

Table II 


Symbol 

Aj, Aj f Aj 

Be 


B. 


B, 


List of Definitions 
Value or Definitions 
0.0205 

Variable coefficient 

0.442 

0.757 


C 


0.396 

General coefficient 


Symbol 

F,(tf) 

fc(tf) 

f'l(tf) 

N 

Ks 

K7 

t 

Subscripts c, s, 1,2,3 

Subscript f 
Subscript i 
Subscript 0 
Subscript t*v 
Subscript (x,x) 

Superscript bar, (as inf) 


Table II (Con’t) 

Value or Definitions 
Cumulative failures, component test 
Cumulative failures, system test 
Cumulative failures, in orbit 
d(F^j(tf))/dt 

d(Fi(tf))/dt 
Number of components 

0.00512 (average failure rate for first 12h of component 
testing) 

N X 10'7 

N X 7.5 X 10-^ 

time in test or in orbit 

Refer to component or system level tests or first, second 
or third ffights, respectively 

Final value 

Interconnect 

Other causes 

Thermal'Vacuum causes 

Decisions D2 and D3 apply; D2 only if ot..y one value 
IS noted 

Average value 




PcW 


N Ag t®« 


( 20 ) 
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On . pe, componnn, basi., this can b. diffaren.iated .o ,iv. a cumula.iva failure rat. of 

failures per component per hour. 

At the end of 187h, the failure rate is ihen 


fc( 187) = 0.000489 


( 22 ) 


tmpCn. ^5™ f 

numb« of feflurea duri^^ "“rlHeraT ^ ^ 


F,(t) = N X 0.000904 t®* 
and the cumulative failure rate per component as 

f,(t) = 0.000904 B- ^ 


(23) 


(24) 


If wc Essumc d constant failure rata over the fir^t i q ^ 

of a test program is relatively eomtant o»« a 1 2h p'eriod “e ci"ltl'e th'. 

failure rate at the beginning of the system test as ’ ^ ^ ^ cumulative 


f,(12) 


1 [ 

12 J 


.(VO 


j2 j 0.000904 Bjfs'-'dt 


(25) 


or 


isU-i; = U.000494 


\4-W/ 


o?rr;rt\« -partng that at the end 

be true at least from the faeVt^Tt^ This should 

components ^ so tha Thet mu« T interconnection of the 

generaUy. some othVr pre^^t interconnecting hardware plus, 

5 <n, to account for this 

ii(12)»fj(12)-^(187) 


• 0.000494 - 0.000487 
?i(12)» 0.000007 . 


(27) 
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TAt nutbci ofr --ponents:;c7ppro;ra^^^^^^ A ion 

of the number of components and designate a term Kg such that luncuon 


fj(12) * N X 0.000007/65 “ N X 10'^ « Kg . 


(28) 






(29) 

or 



Then, taking Aj 

A, ■ ^(12) • ( 12 /'"®!* . Kj(12/‘"®s\ 
as was done with Aj , we may write 

(30) 

If 


(31) 


0 

(32) 


‘'c('cf)"-ijV>2)®’ = A,(‘2i“’'‘’ 

(33) 

then, from (30), (31) and (34), 

(34) 


f,(t,)- [K«(12/'-*>> + y,^fK12/‘'®>']B,(t,/®>-" 

(35) 

and 

f.«.) • [Kj + Q(t,f)l X 02 /' ■“«> B, «/*!■'' 

(36) 


F, 0 ,)-IK 4 + f,(tj()| x(i;)'‘'®>’t®« . 

(37) 
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Within the 39 spacecraft that were used to develop (23), were 31 that were included in the PRC 
data base. Front this data base, and again using a 24h time increment for curve fitting, we can 
describe the orbital performance as 


Fj(t)» NX 0.00288 t®^ 


( 38 ) 


and 


fl(t)« 0.00288 

The average failure rate for the first 12 h is 

1 

fl(12) = ~ J fj(t)dt 
•'o 


(39) 


(40) 


= 0.000642 . (41) 

Since fj (415.2) = 0.000158, there is a significant difference between the failure rates at the end of 
the system-level test program and the beginning of the orbital case. Again, this is not unexpected 
since the orbital case includes failures due to causes other than these one may associate with the 
thermal-vacuum test, e.g. failures due to vibroacoustics. Let us designate these “other cause” 
failure rates as f(,(t). If we assume, pending more complete data analysis, that the failure rates 
may be transmitted from the system test level to the orbital case, then based on the current data 
we may write as in (27) 


fo(12) = fj(12)-fj(415.2) . 


(42) 


Substituting from (41), we have 

fg(12) = 0.000642 - 0.000158 = 0.000484 

If we assume fg to be a function of spacecraft complexity as measured by the number of 
components, we can establish a coefficient K 7 such that 


K 7 « N X 0.000484/65 » N X 7.5 X 10‘^ . 


Also, if we assume that the f^ and f 3 terms arise from similar equations having the same 
exponent, B 3 , we may proceed as in (29). 


In orbital case, we may write 

fo(12) 


JL 

12 






(44) 


( 45 ) 


Substituting the vaiue for K 7 as in ( 44 ), we wrtte 


Ao-fo(I2)X(12)<'-“l).K,(i2)<‘-“l> , 


Then, as in ( 12 ), 


(46) 


fl(t)«(Ao+Ai)Bj 

from which, as was shown in developing ( 34 ), 


(47) 


and 


and 




Fi(ti) = (Ao+Ai)ti®I 


fl(ti) = (Ao+Ai)Bi 


(48) 


(49) 


(50) 


the att^entT^O the form of (49) precludes 

by the thermal-vacuum test program If the A re™ * Aq term which is unaffected 

causes that could be uncovered fy thennal-vacuum ^ u®” ^*““^** occuired due to 

could be achieved would stiU be no greater than * average avaUability that 


/ n -0-273 exp(-0.0086N)] 0^^(J2)^ 

1 0 H 


dt (51) 

cases* developed previously, we can describe performance in orbit for three other 

1 - component test but no system test, 

2 . system test but no component test, and 

3. no test. 

Since there is no actual data for these cases, additional 
pontponent Test but no System Test 


assumptions will be needed* 


program and the failure rates can be expressed 
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( 52 ) 


and 




(53) 


If there were a system level test, the average failure rate during the first 12h, would be, from (37), 

^*^6 + fc MI Bj t^®»’^^ dt 

* + fc (y)l (12)^^’®*^ (12)^®»*^^ . 


fs02)-K6 + f^(t,f) 


(54) 


Some failures would be generated, but we choose to neglectlhem becaiKe they are unassignable. 


propogate (12) forward to the orbital case as though it were 
and (50), we may define the orbital case characteristics as 


fs(tsf> in (48), (49), 


Fl(ti)-N(A„+A,)t,®' 

Fl(t,)-N [k, (12)“-®l^f,(, .)(|2)'‘-®1>] ,“l 

■■ • 

““ ® “V«age of the Bm 1 2h were 


f,(t,f)-f,(12) 

and, from (54), (55) may be rewritten as 

Fi(tj)«N [K7(12)^*"®i^ + (K5+yt,f))(12)^^"®l^]tj®l 

Fi(ti.)*N(f^.(tgf) + Kg + K7J . 

Note that Fj (tj ) cannot be less than 

Fi(t|)-NtK6+K7](l2)^‘-®lItj®I , 


(56) 


(57) 


(58) 
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(59) 


Also, given U and tj , may be defined as 

t,f •(((F,(ti)/(N(12/‘-®l) -K,)/(A, 

SO that any component test duration equal to or larger than tgj- will provide at least the minimum, 
desired U . It is possible that the necessary tgf < 0 m which case no tests will be necessary. It 
is ateo possible that no solution exists in which case U was chosen below the possible minimum. 

2. System Test but no Component Test 

We take the average failure rate of components, had there been component level tests, as the average 
faUure rate for the beginning of the system level test, both over the first 12 h interval. 

(60) 

0 


Substituting in (36), 


W* [K6(12)“'®>’ +K5(12)“'®>^ B, 

(Kj + Kj) (I2)'‘‘®>’ B, 
and 

F,(t,) = N(K5+K6)(12)^*■®*^®^ 

As in (46), Aj = (l|(tjf) (12)^ ^ ^ and, substituting from (42), 

Ai - (K 5 + Kg) (12)^^"®*^ B, (12)^^“®!^ 

Ai "(K 5 +Kg) B, 

and from (47), 

F,(t,). (K7(I2)<‘-“>>+(K5tKjK12)‘^-''«-“l>B,t,‘®>-'*l t,*> 

For a prenleflned U and t . , we can solve for t, as 

t, - (((Fj (tj )/tj ® 1 ) - K 7 ( 1 2)^ ^ 1 ^/((Ks + Kg) ( 1 ^ ‘ ‘ 


(61) 


(62) 


(63) 


(64) 


(65) 
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3. No Test 


In this case we assume that the initial component failure rate characteristics propagate to the system 
test, and the initial system test failure characteristics propagate to the orbital case. As in (60), we 
deflne the initial component level failure characteristics as 

fc(12)»K5 . (66) 

The system level failure characteristics are then described as the average of the fiist 1 2h of hypo- 
thetical system test or, from (61), 

fs(12) « (Kj + Kg) (12)^^"®*^ (12)®* 

f,(12) = K5+Kg . (67) 

Since the failures that these failure rates infer do not take place during tests, they are not con- 
sidered. 

Since (12) is the same as fj(tj) in (48), we may write 

Ai =(K5+Kg)(12)^^"®l^ (68) 


and, substituting into (49), 

Fi(tj) = N[K7(12)^‘‘®l^ + (K5+Kg)(12)^^"®l^ tj^i 

Fi(ti) = N[K5+Kg + K7l (12)^‘”®‘\®» . (69) 

If one has a mission of duration ti and a predefined U , then, if the number of failures found by 
(69) is less than those that produce the necessary U , no test is needed. 


Failure Flow Process and Utility for Multiple Flights 

In order to demonstrate the approach and not to become too cumbersome, we limit ourselves to 
no more than three flights (although the process allows for more). Table III indicates the various 
options that can be exercised 
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Table III 


Available Options 

Test Decisions Flight Decisions 

Component System 

Level, C Level, S Pint, Dj Second, D 2 Third, Dj 

Og 0, - 0 0 

1 , 1 , 1 1 1 

2 2 

3 3 

4 4 

where, under C and S: 

Og indicates no component level testing, 

0 , indicates no system level testing, 

1 ^ indicates component level testing, 

1 j indicates system level testing, 

so that (Og, Oj) would indicate that neither component nor system level tests were done and ( 1 ^, 

0 ,) would indicate that component level tests were done but that there was no system level thermal- 
vacuum test. 

Similarly, under decision Dp D 2 > and D 3 : 

0 indicates no reflight, regardless of the results of the previous flight 

1 indicates reflight if the mission is not accomplished (2,3, and 4 follow from this) 

2 indicates reflight without repair, refurbishment, or retest 

3 indicates reflight with repair and refurbishment but no retest 

4 indicates reflight with repair, refurbishment, and thermal-vacuum retest. 

Since there is no program without at least one flight, the decision for indicated by 0 does not 
exist. 
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We can now consider the decision 
has a number of options. 


matrix (C, S, Dj , D2, D3 ); each one (with the exception of Dj ) 


p«haTl‘ “"s'" 

will be less than the oath n l 1 n m ’ h- **^®r®»'tees that the overall average utility 

be fairly Srtain tha/we^iui ° evolves one flight rather than two. However, we can 

The difficulty lies in the definition of overall, averase utilitv i ^ 

«« *:<«« payload ,o ,ha, of a p«f,c. ona. The pro'hta ” 



Figure 1 1 


Effect of D2 Decisions on Cost and Average Utility 

utm. <" » *" 'O'* 

« the point Ds . 3”^* ‘rn h“J" “ 

It Will hi e.u * I* higher and almost certainly to the right of point ^ 

""nr.'’* ""'"“k'";" 

fonnance duHn. the Hta. „l.ht and L !ucee„“T;i:.= ti'^rp'S,: 


32 


non-<l»nn»l-vKuum related failures as the duration is extended, and the 

the Mcond ni«ht would correct these flaws that would otheiwise have detraded the 
performance of the second flight.) 


hiJLT. f “T' ‘ *" ‘ *' Pt«ess in terms of a mission when 

our goal IS to gather a specific amount of data. 



Figure 12 

Effect of D 2 Decisions on Cost and Data Gathered 

Here, as before, by conducting a particular test program, we airive at point D, having gathered 
some portion of the desired quantity of data, A. If we now simply refly (without any reoairs or 

was gihlreT^pS^ D* Jh ^*h**^h'’“* probably increase the quantity of data that 

was gathered. Point D 2 * 2 would then be above and to the right of point Di . Decisions 3 and 4 

"" 2 ? D2 -4, increasingly to the rStand^bove\)^^^^^^^^^ 

mtuitively determine whether we will cross the line at A, but we would at least m*ove toward it 
AS can be seen in flprre 1 1, i, is posabte to pro,ress to the left as a flight p'oirlTcoSeS.' 

Figures 1 1 and 12 can be thought of as depicting processes important to two different kinds of 

' ‘r”* » *«'■ of performance hwt 

TOght be the case with a communications relay satelUte. Figure 12 could represent a scientific 
V ^ ^ defining the ultraviolet spectral signatures of a certain group of stars In either 

case It IS important to know the instantaneous capabUity of the payload (as might be defined by 

which It appUes, For the purpose of this paper, we will Umit ounclves to 
^ose cases where average utUity is the appUcable criterion. Further, it is applied on a per-flight 

T (designated as U, ) is established by the decisions at 

co!t w! 7h approach, i.c., a specified 0, for minimum 

r andDj with the assumption that 0, doesnot 

fulfill the mission requirements. * 
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From preliminary data analysis, it appears that for cases involving costly payloads, the minimum 
cost programs involve thermal-vacuum testing at both the component and system levels; we then 
discuss tWs case first. Furthermore, it is assumed that flights may be viewed or act upon a payload 
u an additional te^. We therefore can. proceed with .I.e concept of a failure flow process, e.g., if an 
incipient failure doesn’t occur during this flight, it may during a test following the flight or during 
a succeeding flight. 

Flowing from flight 1 to flight 2 without any repair, we have 

<™) 

fl(t,f) = ^A,0:)®2.Aj(12/®2-') 

Aj=f,(t,f)(12)'‘"“2) (72) 

where B2 ~ Bj and tjf is the final flight time of the first flight for the minimal cost-utility 
strategy using component and systems level testing. 

U20) means the average utiUty for flight 2, under decision 2, and U2n\ means the average 
utihty for flight 2, under decision 3. ^ 

Note that these are not utiUties that include the previous flight. 

As before. 


f2«2)* + (12)“ '“2> Bj l2'®2-‘) 


( 73 ) 


Since we currently have no data to speculate on the constant Kg , where Kg is a constant similar 
to K7 but between the first and second flights, we assume that it exists for similar reasons as 
before and assume Kg ■ K7 . This assumption means that we are including a flight interconnect 
type term in our model. It also means that the lift-off has an effect on failures that cannot be 
accounted for. From ( 67 ) we have 


F,(2)(t2)- [fidifltKjl (12)“‘®2’t2®2 (74, 

and 

^2(2) / (1 jjt 

^2f tjf 

This argument generalizes if we consider the various combinations of test options because ft (ti f) 
can be calculated under these options as explained in the previous sections. 


'1 
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FLIGHT 2, OPTION 3 


,nH th u ^ ^ ^ "® detectable in the thermal-vacuum test environment 

and those which are not. Thus we have Ks/2 instead of Kg. environment 

The utility is based on 


and 


hence, 


4m * 

*2f 0 ^ • 


(76) 


(77) 


(78) 


TLlOMl 2 , OPTION 4 

has started over at 1 .0 to begin the flight but with the additional cost of 

^ change in the failure mode. To account for the change 
m the failure mode, we use the mechanism as described before; thus we have * 


or 

where B 2 , * B 2 
As before. 



(79) 

•yA2,(12)®2> = Aj,(12/®2>-‘^ 

(80) 

A2,«fi(tif)(12)^*‘®2s> 

(81) 

^2s(4)(t2s)-tfi(tif)l (12)^*“®2s) ujhr^\ 

(82) 
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and 


''2.(4)«2.)- If, (t,f)l (12)“-*!^ tj, 


(83) 


We have not assumed a component test stage since it appears most likely that only the few com- 
ponents that have exhibited a failure would be retested, and this is much like the process in a 
system level test program. After the test stage, these failures flow into the second flight, and we 
have 


therefore, 


where B 2 = Bj . 
As before, 


and 


Therefore, we have 


^2s(4)(*2sf> “ Aj B 2 t2^^* *^dt2 (84) 

f2.(4)(*2$f)"]TA2(^-)^^“A2 (12)^®-"”^ ; (85) 

A2 = f2s(4)(‘2sfH12)^^“®2) (86) 

K 

‘'2(4)«2) = If2,(4)(‘2.f)+ 2^ 1 (12)“'“2' B, (87) 

F2(4)(*2l"|f2j(4)«2lf)'^Y ’ • (88) 


0j(4)-^ /^'(l-D*)‘''2>(-')"2”dt, 
*2f (> 


This discussion takes care of the calculation for flight 2 after the decision I >2 is made. We now 
turn to the calculations based on the decision D 3 and options ( 2 , 3 , 4 ) before third flight. 


FLIGHT 3, OPTION 2 

Flowing from flight 2 into flight 3 with D 3 * 2, we must base our analysis on each of the three 
failure rates, f 2 ( 2 )(t 2 ^* ^ 2 ( 3 )(* 2 X f 3 ( 4 )(t 2 ), from the previous decision'>making process 4 
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Using the final failure rate f2(2) (t2f)* 


f3(2,2)('3)- If2(j)(t2f) + Kj| (i:/*-*3) ,j«3-‘> 


and 


(90) 


^"3(2.2)(t3)"(f2(2)<*2f) + K8l (12)^'’®3\3®3 

where we assume B3 ■ B2 ■ Bj until we have data to adjust these estimates. Thus, 

/ ^1 

(l-D*)^''3(2.2)(*3))dt^ , (92) 

‘3f t|f+t2f 

wc^Sve" ^ ^ ^2(3)(t:f). and 


K 


^3(3,2)(*3^ ® f^2(3)(^f^ ■*■ 2" J (12)^' 


(93) 


and 


Kg 

P3(3,2)(*3)"lf2(3)(t2f> + y ) 02) 


O-B3) 



(94) 


Then, 


U 


3(3.2) 


'3f 


*2f 


(l.D*)^‘^3(3,2)(t3))dt3 . 


(95) 


If we now consider the case where option 2 for the third flight follows from option 4 for the 
second flight, we use f2(4)(t2), and we have 


Ko 

^3(4,2)03) * tf2(4)02f) y ) (12) 


(I.B3) 


B t ^®3-» 

Bj (3 


(96) 


and 


Kg 

f3(4,2)»3 )-lf 2 ( 4 )<' 2 f)*-I(l:) 

4* 


(I-B3) 



(97) 
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We have for utility 


‘^3(4,2)*,— dt, 

'3f ,2f 

FLIGHT 3, OPTION 3 

Following from flight 2 into flight 3 with option 3, we must base our analysis or calculation of 
utility on each of the three failures rates, f2(2)(*2)> ^2(3)(*2)> f2(4)(*2)* ^^om the previous 

decision making process. These derivations wm be similar to the previous ones, but with an 
adjustment for repair in the utility calculations. 

We have, similar to the previous equations. 


and 

K 

^3(2,3)(*3)“ f^2(2)^*2f^'*‘~ J 

where Kg has been adjusted by a multiple of 1/2 due to one repair in the failure flow process. 
Kg is adjusted by a multiple of (1/2)** (where n is defined as the total number of repairs in the 
process). We use this method to account for learning. We have for utility. 


^3(2,3) 


— f^^l-D*)^^3(2,3)(»3)) 

*3f 0 



( 101 ) 


If option 
Iiave 


3 for the third flight follows from option 3 for the second flight, we use f 2 ( 3 )(t 2 f) and 


^3 (3,3) ^^ 3 ) * f^2(3)^^2f) J 
and 

^3(3,3)<*3>* t^2(3)^*2f>‘*"7 5 02)^^"®^^ tj®^ 

with 


r-*3f 


C3(3,3)*,— / (l-D*/‘'W.3)f3»dt3 . 

*3f 0 


( 102 ) 


(103) 


(104) 
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If option 3 for the third flight follows the failure 
then we use f2(4)(t2f); then, 


flow process from option 4 for the second flight, 


and 


with 


^3(4,3)(*3> • I^‘2(2)^‘2f) + 1 (i:/^"®^) 

“• I ^3f 

^3(4,3) “r- / (I-D*/''3(4,3)(‘3 ))jj, 

* 3 f o ^ 


(105) 


(106) 


(107) 


rtium 3, OFTION4 


Under this decision, we must take Mch of the final faflure rates fw, 1 f r, > . 

f 2 ( 4 )(t 2 f). and process them through a re-test stage and then S^iiS.'‘thVth^'&'’'^', 
Starting with f2(2)(t2f), we have 


‘' 4 ( 2 )(‘ 2 f>’n i''A3,B3st3/‘3>-‘>dt 


3 s 


(108) 


where B3, » 83 . 

If we make derivations similar to equations (79) thru ( 83 ), we have 


^3s(2,4)(*3s) * f^2(2)(t2f)l (12)^*‘®3*> 


and 


(109) 


^3 s( 2,4)(*3 s^ * f^2(2)^*2f^J (12)^^ ^^*^*3$^^* 


Extending f3,(2,4)(t3,) into the third flight, 
( 81 ), we have 


and using a derivation similar 


(110) 

to equations ( 78 ) thru 


^3(2,4)(*3) - 1^35(2, 4)(‘3,f)’^ -T 1 (12)^*‘®3 >b3 


( 111 ) 
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and 




( 112 ) 


(I-B3) 


B 

t 3 


3 


with 


03(2.4) /'’'(1-D*)"’3(2.4)('3»4, . 

*3f -b ^ 

We extend f2(3)(t2f) and f2(4)«2f) Into the system level test and obtain Us(3 4^(tss) and 
^3s(4,4)(*3s) as in equation ( 113 ). From these equations we obtain f3f3 4-j(t3) and 
f3(4,4)(t3) as in equation (111) and ¥3(2, 4)i^) and F3(4,4)(t3) asWequation(112). 

These failure functions yield 


^3(3.4)=— / ^^lW^^(3,4)(t3)) dt 
* 3 f 0 

03(4,4) ' 7 ^ /”(1-D*)<’'3(4.4)«3»d, , 

* 3 f o ^ 


( 114 ) 


( 115 ) 


As a final comment on utility, we mention that their confidence intervals are calculated as in the 
previous section where time aUowances are made for the various repair, refurbish, no repair etc 
options. ’ ' 


For example, if there is no repair and two flights are made, where tj is the time in the first flight 
and t2 is the time for the second flight, then we have the total time interval from (o, tj+t2). 

To calciUate the variability for average utiUty after two flights, we calculate failures using the 
assumption of process dependence at the end of the time (tj+t2). 

If repair is made after the first flight, then we calculate the failure made at t-> . 

Another scheme to place bounds on average utility can be devised by using the Product Limit 
procedure and regression to calculate upper and lower bounds on the failure mode functions F(t) 
To lUwtrate this relationship, we construct the curve F(t) and its upper and lower confidence 
tounds versus average utiHty achievedjsee figure 13 ). Note that as the cumulative number of 
failures increases, the average utility, U, decreases. 


IMU > U,). 0.95, U,< 0,8,*. calculate 90% 

Product Limit confidence bounds, Fy(t), FL(t) by regression using a real data base. We then 
project up to Fu(t) from 0 . 8 , over to F(t), and then down to U at Uj where Pit 0 >Uj)» 
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Average Utility 


Figure 13 

Bounds for Awrage" Utility 


This argument can be reversed to yield a test program which gives the desired utility based on a 
min imal cost thermal-vacuum program. What one does is to reverse the path, find the utility one 
needs to obtain, and then refers to the minimal cost curve for thermal-vacuum test program to 
determine the particular test program. 


RECOMMENDATIONS FOR FUTURE STUDIES 

1 . Study the question of complexity to see how it affects the calcuiation of utility. 

2. Use the science of Information Theory to better quantify ideas about obtaining and using 
information from Space Shuttle experiments. Build guides for management to use in planning 
missions in terms of the actual information needed to complete an experiment. 

3. Apply utility to individual components in an experiment having several components to deter- 
mine the model of overall utility for the total number of components. 

4. Study the process dependent assumption; obtain data to either verify or change this 
assumption. 

5. Gain data from Space Shuttle missions to model and establish the relationships of failure 
between individval flights and to measure the interconnect effects in flight as well as for test. 

6. Trace failure flow data through component and system levels thermal-vacuum tests and on 
into flight. 


41 


j 


7. The timing of testing; is testing prior to flight 1 equivai 'nt to testing between flights? Does 
the tuning matter? 


8 . 


Is there a relationship between complexity and refurbishment costs? 
LANDSAT data might reveal one. 


An examination of the 


9. 


Is there an ultimate ^*ss than 1 .0? Our model and the intuition 

suggests that there is. If so, what is that level and why? 


it is based upon 


10 . 


The cost figures appear to be changing rapidly. An updated version of The STS Reimburse- 
ment Guide IS necessary to calculate accurate costs. Too, since updating is likely, they should 
be easy to change m the computer program. 


11 . 


A more careful look at scientific decision making is needed. We have shifted the model’s 

output toward information that we believe is more attuned to managers’ needs. The question 
now IS whether they will. 
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